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ABSTRACT 

This study examined planning processes and the 
acquisition of expert problem solving skills among individuals of 
different expertise on tasks of increasing complexity. Twelve 
French-speaking adult students of machining (novices) and 12 
French-speaking teachers of machining (experts) in Quebec (Canada) 
were randomly assigned to a simple or complex task. Thinking-aloud 
protocols were collected and analyzed according to a specific coding 
system in order to examine the amount of planning and other 
operations performed by the subjects. No significant differences were 
obtained for the planning variable. However, significant differences 
were* found for the writing and evaluation variables. Also, the 
quality of the performance was linked to the amount of planning done 
by each subject. Possible explanations for the results and future 
directions for research are presented. One bar graph illustrates the 
discussion, and there is a 17-item list of references. (SLD) 
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This study examined planning processes among individuals of different 
expertise on tasks of increasing complexity. Twelve novices and twelve 
experts in the machining domain were assigned randomly to a simple or 
complex task. Think-aloud protocols were collected and analyzed according 
to a specific coding system in order to examine the amount of planning and 
other operations performed by the subjects. No significant results were 
obtained for the planning variable. However, significant results were found 
"> ine writing and evaluation variables. Also, quality of the performance 
• 'as linked to the amount of planning done by each subject. Possible 
explanations for the results and future directions for research are presented. 



The aim of this study was to examine the acquisition of expert problem solving strategies, 
more specifically the planning processes of individuals with different levels of expertise 
performing on tasks of distinct levels of complexity. In addition, the quality of subjects' 
performance was examined and linked to the amount of planning performed by each subject. 

Problem solving has been studied in different domains such as physics, mathematics, and 
computer programming ( Anderson, Greeno, Kline & Neves, 1981; Greeno, 1978; Greeno & 
Simon, 1987 ; Larkin, McDermott, Simon & Simon; and Newell & Simon, 1972). One issue of 
particular interest has been the study of strategies and how they are acquired. One may define 
"strategy" as a systematic process used by different individuals to solve a problem. Strategies can 
be either general or specific to a domain. General problem solving strategies such as planning tend 
to be used more frequently in unfamiliar situations and by unexperienced individuals (Anderson et 
al, 1981, Greeno & Simon, 1987). 

Generally, planning may be defined as the anticipation and organization of several steps 
necessary for the solution of a problem (Breuleux. 1990; Hoc, 1987/88). Goals and plans have 
been used as the basis fc* explaining human behavior (Miller, Galanter & Pribram, 1960). 
Planning as a general strategy is very useful in order to cope with changing and complex 
situations. As a result, some researchers have investigated planning processes and the effect of 
expertise on planning (Goldin & Hayes-Roth , 1980; Hayes-Roth & Hayes-Roth, 1979; Davies, 
1990; and Sebillotte, 1988). Goidin and Hayes-Roth (1980) found in their study on planning 
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processes that good planners revised and assessed more often their previous decisions and were 
more likely to consider alternative solutions since they were more sensitive to constraints- Davies 
(1990) found that experts in the programming field seemed to expand their plans in a depth-first 
fashion, whereas other studies suggested the more frequent use of a breadth-first expansion for 
experts in order to be able to deal with constraints such as interactions between sub-goals (Flower 
& Hayes, 1980; Hoc, 1987/88; Jeffries, Turner, Poison, and Atwood, 1981). Furthermore, 
Hayes (1987) examined the planning strategies used by expert machinists. Results showed that 
experts were able to avoid constraints and were able to divide the problem into independent sub- 
problems, solve and merge them after. 

However, relatively few studies in the problem solving literature in cognitive psychology 
have studied planning processes. Most studies have tended to provide either a) a detailed 
description of problem solving strategies using a very limited number of cases, or b) relatively 
superficial descriptions of problem solving strategies using statistical tests and larger number of 
cases. Very few studies have combined detailed analyses of planning processes with the study of 
the expertise/task complexity effect on planning. 

The main contribution of this study was the use of detailed analyses of think-aloud protocols 
in order to obtain measures concerning the planning processes of individuals involved in this 
study. This was done in order to achieve a better understanding of planning skill acquisition as a 
function of expertise and task complexity. 

Task analysis: 

Machining tasks involve transformation operations where an initial situation 'raw material 
and technical drawing) is presented and the machinist has to perform a sequence of actions such as 
applying the appropriate transformation operators on the initial state (i.e., raw metal piece) to 
obtain the goal state (i.e., the product). The machining curriculum developed recently by the 
Quebec Ministry of Education identifies the construction of the "gamme d'usinage" or procedure 
specification as the focus of training. Such a procedure specification involves the breakdown of 
the task into steps, sub-steps and operations (within sub-steps). The feasibility and optimization of 
the order of steps, sub-steps and operations are the main concerns in the machining domain. 
Machinists must be able to interpret the technical drawing and establish the procedure to obtain it 
They must also anticipate and identify precisely the tools required for each operation. Also, they 
have to represent the shape of an object at different points on the procedure specification sheets. 

The task of specifying a machining procedure involves several types of operations. The 
important ones in this study belonged to three major categories: writing operations, control and 
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monitoring operations, and calculation operations. Writing operations were the operations used to 
fill in the procedure specification sheets. Control and monitoring operations corresponded to 
problem solving operations such as planning, evaluation, elaboration, reasoning, and clarification. 
Calculation operations corresponded to the subject calculating different values required by the task. 

Hypotheses: 

Based on the assumptions and findings stated by previous mentioned researchers, concerning 
the more frequent use of general problem solving strategies such as planning in novel situations 
and by novices compared to experts, two hypotheses were forwarded: 1) Experts will plan less 
than novices and; 2) both experts and novices will plan less in the simple task than in the more 
complex task. 

Methodology 

Subjects 

Twelve French speaking adult students were identified as novices. The experts were twelve 
French speaking teachers in the machining domain teaching at the D.E.P level (diploma of 
professional studies). In addition, experts had industrial experience. Experts were from different 
regions of Quebec. 

Material 

Two industrial drawings of increasing level of complexity were used. These drawings were 
provided by a consultant domain expert. The simple drawing "Drawing 1" required the subject to 
produce one piece. The complex drawing "Drawing 2" required the production of 100 pieces. 
Answer sheets were provided. 

Procedure 

A short description of the general goal of the study was given to each subject. Then two 
warm-up tasks were presented. The objective of these warm-up tasks was to encourage and ease 
verbalization. After the completion of the warm-up tasks, each individual was given one of the 
two technical drawings and answer sheets were provided as well. Subjects had to write down all 
the steps required in order to produce the specific machining part shown on the drawing. Subjects 
were asked to verbalize their thoughts throughout the task. All verbalizations were audiotaped. 
Once the task was finished, subjects were asked some clarifying questions concerning their 
procedures such as "could you explain the order of the steps involved in this task?". Answers to 
these questions were audiotaped. 
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Data analysis 

All audiotape data was transcribed. Transcripts were segmented and each segment was 
analyzed according to a reliable well defined coding scheme (Breuleux, 1990). Such a coding 
system would allow to observe: relationships between goals in plans, the amount of planning 
done, the amount of plan revision and the structural properties of plans constructed. In this study, 
only the amount of planning was considered. 

Each segment (i.e., verbalization that corresponds roughly o a sentence as defined by 
Grevisse (1986)) was coded as belonging to a specific semantic category (i.e., Goals, Writing, 
Calculation, Elaboration, Statement, Clarification, Reasoning, Rereading, Evaluation, Auto- 
Evaluation, Incomplete, Supposition, and Repetition) and then several segments were grouped into 
episodes. The main types of episodes were: 1) Planning, 2) Writing, 3) Calculation, 4) 
Evaluation, 5) Elaboration, 6) Evaluation and Elaboration, 7) Rereading, and 8) Observation. 

Furthermore, performance by each subject was rated by the expert consultant, according to 
the typical criteria used in technical schools. These two criteria were whether or not the procedure 
was feasible (i.e., logic of the ordering of steps in the subject's procedure) and optimal (i.e., 
whether or not the chosen procedure was the most suitable for the specific task). A procedure 
needs to be identified as feasible before being considered as optimal. 

Design and statistical analyses 

The independent variables were expertise (i.e., novice/expert) and level of complexity (i.e., 
"Drawing 1" , simple task and "Drawing 2" , more complex task). The dependent variables 
consisted of the proportion of segments within episodes identified as : Planning, Evaluation, 
Writing, Calculation, Observation, Elaboration, Evaluation and Elaboration, Rereading, and Non- 
Coded. Analyses of variance were performed. 

Results 

Planning 

Experts in both task conditions seemed to plan more than novices, with experts in the more 
complex task condition, Drawing 2, appearing to plan the most. Novices in both task conditions 
seemed to plan equally. However, there was no significant task complexity effect (F (1,20) = 
0.565, p < 0.461 ) and no significant difference (although closer to significance 

compared to the task complexity effect) between experts and novices in terms of the amount of 
planning (F (1,20) = 0.224, p < 0.151). In addition, there was no significant interaction of 
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expertise by level of task complexity in terms of planning (F (1,20) =0.604, p < 0.446)(see 
Fig.l). 

0.25 t 




Experts Novices Experts Novices 
Drawing 1 Drawing 2 

Figure 1. Mean proportion of segments in planning episodes for ail conditions 

Furthermore, quality of performance was linked to the amount of planning performed by 
each individual- No statistical analyses were done concerning the individual amount of planning, 
however, 1) overall experts seemed to do better than novices and; 2) the best performers in r" 
experimental groups seemed to plan the least. For example, in the protocol of the expert SI ' u 
about 5% of the total number of segments were coded as belonging to planning episodes, and for 
experts S 1 and S 1 2, 1 6%. However for the expert S4, the amount of planning was 23%. 
Overall, S 1 and S 10 seemed to be the best two performers. They also seemed to plan the least. 

Evaluation 

Averages seemed to be greater for novices in both task conditions, particularly in the complex 
task. Also, experts seemed to evaluate equally often across both task conditions. The effect of 
task complexity on the amount of evaluation was close to the level of significance (F (1 ,20) = 
4.278, p < 0.052). A significant expertise effect was found for the amount of evaluation 
performed by subjects (F (1,20) = 9.308, p < 0.006), with novices making evaluation statements 
more often than experts. However, the interaction effect of expertise by level of task complexity 
was not significant (F (1,20) * 4.025, p < 0.06). 

Writing 
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The amount of writing seemed to be greater in the simple task for both experts and novices 
than in the complex task (Drawing 2). There was a statistically significant effect of task complexity 
on the amount of writing found in subjects' protocols (F (1,20) = 4.855, p < 0.039), with subjects 
in simple task (Drawing 1) writing more than subjects in in the complex task (Drawing 2). 
However there was no significant difference between the experts and the novices (F (1,20) = 
0.018, p < 0.894) and the interaction effect of expertise by level of task complexity was not found 
to be significant (F (1,20) - 0.586, p < 0.453). 

Other variables 

No significant effects were found for the other variables. However, there were visible trends 
in the data. Experts systematically seemed to do less: 1) evaluation and elaboration, 2) elaboration, 
3) observation, and 4) calculation, than novices. Experts and novices seemed to do less: 1) 
evaluation and elaboration, 2) elaboration, and 3) observation, in the simple task than in the 
complex task. Finally, experts and novices appeared to do more calculation in the simple task than 
in the complex task. 

Discussion 

Planning 

Several possible explanations for the lack of differences seen between novices and experts in 
terms of planning can be offered such as: a ceiling effect for the novices; the selection of experts; 
the rigidity of the coding procedure; verbalization difficulties; and the low number of subjects per 
group. 

One explanation relative to the between-group trend obtained in this study could be that 
experts needed to plan more as the level of complexity increased and that novices experienced a 
"ceiling" effect since their experience and knowledge are quite limited compared to the experts. 

On the other hand, when looking at the quality of individual performance, one can see that 
within each group of expertise, there appears to be considerable variability in performance, and this 
could decrease the homogeneity of groups. This could account in part for the lack of significant 
differences between groups for planning. 

The quality of the verbalizations is another factor that might have limited the present study. 
Novices seemed to have had more difficulty verbalizing their thoughts, and these difficulties have 
probably affected the quantity and quality of the data on planning processes. These subjects 
should have been screened more carefully after the warm-up tasks. 
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Finally, the coding procedure for identifying episodes should be refined by taking into 
account the content of consecutive goal statements to be able to identify as planning episodes only 
the sequence? of operations that represent the construction or modification of coherent plan 
structures (see Breuleux, 1991, 1990). 

Although individual planning differences were not tested statistically, the observed trend can 
be described as more compatible with the first hypothesis of this study, and with the assumption 
that experts plan less. 

Evaluation and writing 

In this study, novices, particularly in the more complex task, were found to spend a greater 
amount of time evaluating their procedures. One possible explanation may be that in a complex task 
situation, novices, not possessing planning processes as elaborated as experts, may need to 
evaluate more often in order to correct the different operations in their procedures. 

For the writing variable, subjects regardless of their level of expertise spent significantly 
more time writing for the simple task. One possible explanation could be that subjects did more 
writing in the simple task because they did not have to engage in considerable planning although 
this explanation has its limitations. 

Directions for future research 

Two types of suggestions can be offered for future research. First, longitudinal studies 
analyzing planning processes among individuals belonging to the same level of expertise would 
help clarify the acquisition of expertiie. This procedure would also allow a greater homogeneity. 
Second, a combination of cross-sectio/tal and longitudinal studies would be helpful since this 
combination would compensate for the imitations found in both types of studies. 
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